Apparatus, method and system for medical image-based radiotherapy planning

ABSTRACT

A medical image processing apparatus includes: a receiver which receives target image data that is including a first perspective image and a second perspective image, photographing an object from different directions; a first acquirer which acquires positional information of a specified point on the first perspective image; a second acquirer which acquires positional information of a candidate point on the second perspective image, the candidate point corresponding to the specified point; and a generator which generates a first enlarged image obtained by enlarging a part of the first perspective image neighboring the specified point and a second enlarged image obtained by enlarging a part of the second perspective image neighboring the candidate point.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromthe prior Japanese Patent Application No. 2013-163425, filed on Aug. 6,2013; the entire contents of which are incorporated herein by reference.

FIELD

Embodiments relate to a medical image processing apparatus, a medicalimage processing method, and a radiotherapy system.

BACKGROUND

Radiotherapy is a therapeutic approach that irradiates a lesion in abody of a patient (object under test) with a radioactive ray to destroythe lesion. At this time, normal tissues are also destroyed unless anaccurate position of the lesion is irradiated with a radioactive ray.

Therefore, at the time of therapeutic planning, the position of thepatient is determined first, then a perspective image in the body of thepatient is photographed in advance by CT (Computed Tomography)apparatus, the position of the lesion in the body of the patient isfigured out three dimensionally from the perspective image, and then adirection of irradiation and a strength of the irradiation of theradioactive ray, which reduce the probability of irradiation of thenormal tissue on the basis of the position of the lesion, aredetermined. At the time of therapy, a new perspective image of thepatient is photographed, and a medical doctor (user) inputscorresponding points on the new perspective image and the perspectiveimage taken at the time of therapeutic planning, performs positionalalignment of the patient according to the therapeutic planning, andirradiates the lesion with a radioactive ray.

However, since the therapy is repeated by a plurality of times, thepositional alignment of the patient needs to be performed at everytherapy. In addition, since the patient needs to maintain the sameposture from the positional alignment until the termination of theirradiation of the radioactive ray, the patient suffers from asignificant burden. Therefore, this positional alignment is desired tobe performed accurately in a short time.

Accordingly, it is an object of the invention to provide a medical imageprocessing apparatus, medical image processing method, and aradiotherapy system which assist a user to facilitate input ofcorresponding points for the positional alignment on a plurality ofphotographed perspective images of the object under test.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view illustrating a position of an object undertest at the time of therapeutic planning.

FIG. 2 is a perspective view illustrating a position of the object undertreatment.

FIG. 3 is a drawing illustrating perspective images taken from twodifferent directions.

FIG. 4 is a drawing illustrating an epipolar plane and an epipolar lineadded to FIG. 2.

FIGS. 5A to 5H are drawings illustrating displayed screens on a display.

FIG. 6 is a block diagram of a medical image processing apparatusaccording to a first embodiment.

FIG. 7 is a flowchart of a medical image processing apparatus accordingto the first embodiment.

FIG. 8 is a drawing illustrating relative locations of radiationsources.

FIG. 9 is a drawing illustrating an epipolar plane added to FIG. 8.

FIG. 10 is a drawing illustrating the epipolar line and a searchingrange.

FIGS. 11A and 11B are drawings illustrating projecting surfaces and FIG.11C is a diagram illustrating relationship among the front image, theside image, and the epipolar line.

FIG. 12A and FIG. 12B are drawings illustrating examples of HLcharacteristic patterns.

FIG. 13 is a drawing illustrating a searching range including theepipolar line in a second perspective image.

FIG. 14 is a drawing of a HL characteristic calculation window.

FIG. 15 is a block diagram of a medical image processing apparatusaccording to the second embodiment.

FIG. 16 is a flowchart of a medical image processing apparatus accordingto the second embodiment.

FIG. 17 is a drawing of a display screen of a display of the secondembodiment.

FIG. 18 is a drawing of a display screen of a display of a firstmodification.

FIG. 19 is a drawing of a display screen of a display of a secondmodification.

FIG. 20 is a drawing of a display screen of a display of a thirdmodification.

FIG. 21 is another example of the display screen of the display of thethird modification.

FIGS. 22A and 22B are drawings of a display screen of a display of afourth modification.

FIG. 23 is another example of the display screen of the display of thefourth modification.

FIG. 24 is a block diagram showing an example of hardware constructionof the medical image processing apparatus.

DETAILED DESCRIPTION

According to embodiments, a medical image processing apparatus accordingto the embodiments includes: a receiver which receives target image datathat is including a first perspective image and a second perspectiveimage, photographing an object from different directions; a firstacquirer which acquires positional information of a specified point onthe first perspective image; a second acquirer which acquires positionalinformation of a candidate point on the second perspective image, thecandidate point corresponding to the specified point; and a generatorwhich generates a first enlarged image obtained by enlarging a part ofthe first perspective image neighboring the specified point and a secondenlarged image obtained by enlarging a part of the second perspectiveimage neighboring the candidate point.

Referring now to the drawings, a medical image processing apparatus 1 ofthe embodiments of the invention will be described.

The medical image processing apparatus 1 of the embodiments is used intherapies on the basis of radioactive ray therapy, photon therapy, andparticle radiation therapy, and is used as a positioning apparatus usingperspective images obtained by photographing an object under test(patient) 2. The term “perspective image” means images obtained byphotographing in the body of an object under test 2 by using aphotographing apparatus such as an X-ray photographing apparatus, a CTapparatus, or an MRI (Magnetic Resonance Imaging) apparatus.

The medical image processing apparatus 1 of the embodiments will bedescribed with reference to FIG. 1 to FIG. 5 below.

First of all, as illustrated in FIG. 1, a user (medical doctor) lays anobject under test down on a bed at the time of therapeutic planning. Inthis laid state, the position of a lesion of the object under test 2 ina three-dimensional space is expressed by V1 (X1, Y1, Z1). The userphotographs the object under test 2 from two directions different fromeach other and acquires the perspective images (for example, X-rayimages).

Subsequently, as illustrated in FIG. 2, the user lays the object undertest 2 down on the bed again at the time of therapy. In this laid state,the position of the lesion in the body of the object under test 2 in athree-dimensional space is expressed by V2 (X2, Y2, Z2). The userphotographs the object under test 2 from two directions different fromeach other again and acquires the perspective images (for example, X-rayimages).

The user manually specifies a plurality of corresponding points in theimages, which are anatomically identical, from the perspective imagephotographed at the time of the therapeutic planning and the perspectiveimage photographed at the time of the therapy, obtains a displacement inposition of the patient at the time immediately before the therapy fromthe time of the therapeutic planning and, on the basis of the obtaineddisplacement, moves the bed to achieve the positional alignment of theobject under test 2.

Since the images used here are the perspective images as describedabove, an occluding contour such as that existing in images photographedby an optical camera does not exist, and hence a front-and-rearrelationship in the body cannot be figured out easily. Therefore, theuser can hardly judge the position of the lesion, which is anatomicallyidentical, by using the perspective images of the object under test 2photographed from two directions, that is, from the front and the side.For example, as illustrated in FIG. 3, the X-ray photographing apparatusincludes a first projecting surface F1 and a second projecting surfaceF2 which are paired with a first radiation source S1 and a secondradiation source S2, and generates a front perspective image(hereinafter referred to simply as “front image”) and a side perspectiveimage (hereinafter referred to simply as “side image”) of the objectunder test 2. However, it is difficult to find a point v1 and a pointv2, which indicate the position of a lesion V1 in the front image andthe side image.

Therefore, in the method of the related art, the point v2 of the sideimage corresponding to the point v1 on the front image exists on theepipolar line L obtained from an epipolar constraint on the side imageas illustrated in FIG. 4. Therefore, by using this geometricrelationship, the epipolar line L is displayed on the side image toassist the user to input the point v2 at the anatomically identicalposition. However, in this method of the related art, there is a problemthat even though a range that may become a candidate of thecorresponding point can be narrowed down from the entire side image tothe straight line L, a trouble of searching the corresponding point onthe straight line L remains.

In view of the above, in the medical image processing apparatus 1 of theembodiment, when obtaining candidate points EP21, EP22, and EP23, ascandidates of the corresponding point on the side image C2, whichcorresponds to the specified point P11 of the front image C1 at the timeof therapeutic planning, the candidate points EP21, EP22, and EP23 ofthe side image C2 are obtained on the basis of the epipolar line L, andenlarged images E21, E22, and E23 of images of portions in the vicinityof the candidate points EP21, EP22, and EP23 are generated asillustrated in FIG. 5, whereby the user can easily input thecorresponding point. In the case where the P11 is determined as thespecified point, P21 corresponds to EP21, and hence the EP31 takes thesame position as EP21 in the image coordinate also in the enlargedimage. The points P21 and EP21 are the identical point on the same imagecoordinate, and hence both points are the candidate point. In the samemanner, in the case of obtaining a corresponding point which correspondsto the specified point P of the front image C1 at the time of thetherapeutic planning on the front image C3 at the time of the therapy,the candidate point EP of the corresponding point of the front image C3at the time of the therapy is obtained, and an enlarged image E31 of theimage of a portion in the vicinity of the candidate point EP isgenerated, so that the user can easily input the corresponding point.The description given above is an example only, and the specified pointmay be input to any one of C1 to C4.

First Embodiment

Referring now to FIG. 5 to FIG. 14, a medical image processing apparatus1 of a first embodiment will be described below.

A configuration of a medical image processing apparatus 1 will bedescribed on the basis of a block diagram in FIG. 6. The medical imageprocessing apparatus 1 includes a first acquiring unit 11, a secondacquiring unit 12, a generating unit 13, and a display 14.

The first acquiring unit 11 acquires a specified point P input in oneperspective image which is a photograph of the object under test 2, andoutputs the one perspective image and the specified point P to thesecond acquiring unit 12 and the generating unit 13.

The second acquiring unit 12 acquires the one perspective image from thefirst acquiring unit 11 and the specified point P, obtains a candidatepoint EP corresponding to the specified point P on another perspectiveimage different from the one perspective image, and outputs the obtainedcandidate point EP to the generating unit 13.

The generating unit 13 generates an enlarged image which is an enlargedimage of a portion in the vicinity of the candidate point EP in theanother perspective image.

The display 14 is a display or the like, and is configured to displaythe one perspective image, the another perspective image, and theenlarged image side by side. An example of the image displayed by thedisplay 14 is illustrated in FIG. 5.

In FIG. 5, reference sign “W1” denotes a window of the image of thedisplay 14.

An image “C1” represents a first perspective image (front image of ahead section of the object under test 2) photographed in a direction Aof the first radiation source S1 at a time (for example, at the time ofthe therapeutic planning) displayed on an upper left side of the windowW1.

An image “C2” represents a second perspective image (a side image of thehead of the object under test 2) photographed in a direction B of thesecond radiation source S2 at a time 1 and displayed on a lower leftside of the window W1. The display 14 does not display the epipolar lineL, which will be described later, on the second perspective image C2.

An image “C3” represents a third perspective image (a front image of thehead of the object under test 2) photographed in the direction A of thefirst radiation source S1 at a time 2 (For example, at the time oftherapy) and displayed on an upper center of the window W1.

An image “C4” represents a fourth perspective image (a side image of thehead of the object under test 2) photographed in the direction B of thesecond radiation source S2 at a time 2 and displayed on a lower centerof the window W1.

An image “E11” represents an enlarged image having a center at thespecified point P on the first perspective image C1 generated by thegenerating unit 12 and being displayed on the upper right side of thewindow W1.

Images “E21” to “E23” represent enlarged images having centers at thethree candidate points EP on the second perspective image C2 generatedby the generating unit 12 and being displayed on the lower right side ofthe window W1.

An image “E31” is an enlarged image having a center at a candidate pointEP on the third perspective image C3 generated by the generating unit 12and being displayed on the upper right side of the window W1.

Images “E41” to “E43” are enlarged images each having a center at eachof the three candidate points EP on the fourth perspective image C4generated by the generating unit 12 and being displayed on the lowerright side of the window W1.

The medical image processing apparatus 1 performs a method of obtaininga candidate point EP on the second perspective image (side image) C2corresponding to the specified point P on the first perspective image(front image) C1 of the object under test 2 at the time of thetherapeutic planning, a method of obtaining a candidate point EP on thethird perspective image (front image) C3 at the time of the therapycorresponding to the specified point P on the first perspective image(front image) C1 at the time of therapeutic planning, and a method ofobtaining a candidate point EP on the fourth perspective image (sideimage) C4 at the time of the therapy corresponding to the specifiedpoint P of the second perspective image (side image) C2 at the time ofthe therapeutic planning separately, and hence these methods will bedescribed separately as illustrated in FIG. 5. FIG. 5A is a front imageat the time of therapeutic planning. FIG. 5B is a side image at the timeof the therapeutic planning. FIG. 5C is the front image at the time ofthe therapeutic planning. FIG. 5D is a side image at the time of thetherapeutic planning. FIG. 5E is an enlarged image of the front image atthe time of the therapeutic planning. FIG. 5F shows enlarged images fromthe front image at the time of the therapeutic planning. FIG. 5G is anenlarged image of a front image at the time of therapy. FIG. 5H showsenlarged images from a side image at the time of the therapy.

First of all, a method of obtaining the candidate point EP on the secondperspective image C2 corresponding to the specified point P on the firstperspective image C1 of the object under test 2 at the time of thetherapeutic planning will be described.

The first acquiring unit 11 acquires a plurality of perspective imagesfrom an X-ray photographing apparatus or the like, or acquires aperspective images photographed in the past from a memory device.Alternatively, a point input at the time of the therapy in the past maybe memorized in the memory device and acquired therefrom.

The plurality of perspective images are perspective images of the objectunder test 2 photographed by irradiating the object under test 2 with anX-ray from an X-ray source of the X-ray photographing apparatus from twodirections different from each other at the same time 1 (at the time ofthe therapeutic planning) and, in this case, correspond to the frontimage C1 and the side image C2. An example of the X-ray photographingapparatus 7 which photographs the perspective images from two directionsdifferent from each other is illustrated in FIG. 8.

As illustrated in FIG. 8, the X-ray photographing apparatus 7 includes afirst projecting surface F1 and a second projecting surface F2 that arepaired with the first radiation source S1 and the second radiationsource S2 respectively in a three-dimensional space on XYZ axes. Theprojecting surfaces F1 and F2 are each a FPD (Flat Panel Detector). AnX-ray output from the first radiation source S1 passes through theinterior of the object under test 2, reaches the first projectingsurface F1, and generates the first perspective image (front image) C1on the basis of energy of the passing X-ray. In the same manner, anX-ray output from the second radiation source S2 reaches the secondprojecting surface F2 and generates the second perspective image (sideimage) C2. In the X-ray photographing apparatus 7, obtained in advanceare: an XYZ axes coordinate system of a three-dimensional spacecorrected or adjusted in respect of photographing position and definedby an apparatus system; and a transparent projection matrix forperforming a coordinate conversion to and from two-dimensionalcoordinate systems of the respective projecting surfaces F1 and F2.

A user (for example, a medical doctor) specifies a specified point P onthe first perspective image C1 displayed on the display 14, and thefirst acquiring unit 11 acquires v1 (x1, y1), which is a two-dimensionalcoordinate position (positional information) on the first projectingsurface F1 relating to the specified point P. As a method of specifyingthe specified point P, for example, the user moves and adjusts aposition of a cursor of a mouse to come to a position to be specified,on the first perspective image C1, and then presses a button of themouse. If the display 14 is a display having a touch panel, a positiontouched by a finger or the like instead of the mouse may be determinedas the coordinate position of the specified point P.

The second acquiring unit 12 acquires the first perspective image C1, onwhich the specified point P is specified, and the second perspectiveimage C2, from the first acquiring unit 11. The second acquiring unit 12obtains a two-dimensional position v2 (x2, y2) of the candidate pointEP, which corresponds to the two-dimensional position v1 (x1, y1) of thespecified point P on the first perspective image C1 (first projectingsurface F1), on the second perspective image C2 (second projectingsurface F2); and outputs the result to the generating unit 13. A methodof obtaining the two-dimensional position v2 (x2, y2) of the candidatepoint EP will be described.

As illustrated in FIG. 9, when a specified point P on the firstperspective image C1 (first projecting surface F1) is projected on thesecond perspective image C2 (second projecting surface F2), a straightline L is obtained. This straight line L is acquired from an epipolarconstrain. FIG. 9 illustrates an epipolar plane M of the epipolarconstraint as added to the X-ray photographing apparatus 7 in FIG. 8,and the epipolar plane M is a plane including three points; the firstradiation source S1, the second radiation source S2, and the specifiedpoint P on the first projecting image C1 in a three-dimensional spaceXYZ. The straight line L, along which the second projecting surface F2and the epipolar plane M intersect, is identical to a straight line(epipolar line L) obtained by projecting the specified point P on thesecond perspective image C2. The second acquiring unit 12 obtains theepipolar line L by applying coordinate conversion to the two-dimensionalcoordinate v1 (x1, y1) of the specified point P on the first projectingsurface F1 by use of the transparent projection matrix described above.

As illustrated in FIG. 10, the second acquiring unit 12 sets aband-shaped range including the epipolar line L and having a certainwidth on the second perspective image C2 as the searching range 3, andfinds the two-dimensional position v2(x2, y2) of the candidate point EPin the searching range 3. A method of finding the two-dimensionalposition v2 (x2, y2) of the candidate point EP in the searching range 3is illustrated in FIG. 11 to FIG. 14.

The perspective image projected on the projecting surface is originally,transparent projection. However, when considering like a parallelprojection, a position (apex) where the object under test 2 is incontact with the epipolar plane M is projected on the epipolar line L asa characteristic point on the first projecting surface F1 and the secondprojecting surface F2. For example, FIG. 11A illustrates the firstprojecting surface F1 and the specified point P, FIG. 11B illustratesthe second projecting surface F2 and the candidate point EP, and FIG.11C illustrates a state in which the epipolar line L is in contact withthe front image C1 of a head region of the object under test 2 and witha lower end of a chin of the side view C2, and an edge of the chin(characteristic point) appears on the epipolar line L in each of thefront image C1 and the side view C2.

Therefore, the second acquiring unit 12 finds the characteristic pointin the searching range 3 including the epipolar line L by an edgestrength, and the characteristic point is determined as the candidatepoint EP. The second acquiring unit 12 uses a Haar-like (hereinafter,referred to as “HL”) characteristic amount as an edge strength forfinding the characteristic point (edge). The term “HL characteristicamount” is a scalar amount obtained as a differential value from anaverage brightness in a rectangular window 5, and the value indicatesthe strength of the brightness gradient.

First of all, as illustrated in FIG. 12, the second acquiring unit 12sets the rectangular window 5 having a certain size, and a HLcharacteristic calculation window 6 having one HL characteristic pattern4 arranged in the rectangular window 5. The user sets a plurality of theHL characteristic patterns 4 as illustrated in FIG. 12A and FIG. 12B.

Subsequently, as illustrated in FIG. 13, the second acquiring unit 12rotates the HL characteristic calculation window 6 having a center at agiven point in the searching range 3 (this point is at an arbitraryposition) in accordance with the inclination of the epipolar line L, asneeded; and calculates the HL characteristic amounts at thecorresponding point. Also, the HL characteristic patterns may beselected in accordance with the inclination of the epipolar line L. Thesecond acquiring unit 12 determines a position having the largest HLcharacteristic amount (edge strength) from among the obtained HLcharacteristic amounts respectively for the points (positions) in thesearching range 3, as a two-dimensional position v2 (x2, y2) of thecandidate point EP. The second acquiring unit 12 may determine theposition having the HL characteristic amount (edge strength) of thelargest absolute value from among the obtained HL characteristicamounts, as the two-dimensional position v2 (x2, y2) of the candidatepoint EP. According to the method of calculating the HL characteristicamount (edge strength), the characteristic points are preferablyprojected as edges in a normal direction of the epipolar line L.

Subsequently, a first modification on finding the candidate point EP, inwhich the second acquiring unit 12 obtains the two-dimensional positionv2 (x2, y2) of the candidate point EP from the searching range 3, willbe described. As illustrated in FIG. 14, the second acquiring unit 12may obtain the candidate point EP by arranging a plurality of the HLcharacteristic patterns 4 in the HL characteristic calculation window 6.

A second modification will be described. The second acquiring unit 12may divide the searching range 3 into a plurality of areas each having apredetermined length of the epipolar line L, and obtain the candidatepoint EP in the same manner as described above in each of the dividedareas. Accordingly, even in a state in which a plurality of edges havinga large edge strength are concentrated, the candidate points EP may beobtained for the respective divided areas without concentrating only onthe concentrated portion.

A third modification will be described. The second acquiring unit 12obtains the HL characteristic amount in advance by using the HLcharacteristic calculation window 6 having a center at the specifiedpoint P of the first perspective image C1 and being rotated inaccordance with the epipolar line L of the first perspective image C1.Subsequently, the second acquiring unit 12 may calculate the HLcharacteristic amounts for points in the searching range 3 in the secondperspective image C2, and use a point most similar in the HLcharacteristic amount to that acquired from the specified point P, asthe candidate point EP.

A fourth modification will be described. The second acquiring unit 12may use points ranging from a point having the highest HL characteristicamount (edge strength) to a point having the Nth (N>1) largest HLcharacteristic amount (edge strength), as the candidate points EP. Whena dynamic range of the first perspective image C1 is different from thatof the second perspective image C2, the image is normalized.

A fifth modification will be described. The description has been givenby using the HL characteristic amount. However, higher similarity may bedecided as follows: similarity in histograms as probabilitydistributions between two points is taken, and the smaller theBhattacharyya distance is, the higher the similarity becomes.

A sixth modification will be described. It is also possible to search arectangular window in which the similarity on the epipolar line L on thesecond perspective image C2 is the highest by using pixel values in therectangular window of a certain size having a center at the specifiedpoint P on the first perspective image C1. A normalizationcross-correlation is used for the similarities. Other imagecharacteristic amounts may be used for the similarities. For example,the smaller the distance between vectors, the Higher the similarity isset if the similarity amount is a vector type characteristic amount.

A seventh modification will be described. The second acquiring unit 12may modify an obtained candidate point EP to a point on the epipolarline L located at a nearest to the candidate point EP.

Subsequently, a method of obtaining a candidate point EP of the thirdperspective image C3 at the time of the therapy or the like, whichcorresponds to a specified point P of the first perspective image C1 atthe time of the therapeutic planning or the like, for example, will bedescribed.

The first acquiring unit 11 acquires the first perspective image (frontimage) C1 by using the X-ray photographing apparatus 7 illustrated inFIG. 8 at a certain time 1 (at the time of the therapeutic planning).The specified point P is specified by the user on the first perspectiveimage C1 displayed on the display 14.

The second acquiring unit 12 acquires the third perspective image (frontimage) C3 photographed at a different time 2 (at the time of thetherapy) from the substantially same direction as the first perspectiveimage C1 by using the same X-ray photographing apparatus illustrated inFIG. 8. Unlike the perspective images (front image and the side image)from two directions different from each other, the acquired image is theperspective image photographed from the same direction, and hence thereis no epipolar constraint. Therefore, a method of obtaining thecandidate point EP on the third perspective image C3 corresponding tothe specified point P on the first perspective image C1 will bedescribed anew.

The rectangular window having the highest similarity is searched on thethird perspective image C3 by using pixel values (for example, luminancevalues) within the rectangular window having a size of a certain rangehaving a center at the specified point P on the first perspective imageC1. A normalization cross-correlation is used for the similarities.Other image characteristic amounts may be used for the similarities. Forexample, the smaller the distance between vectors, the Higher thesimilarity is set if the similarity amount is a vector typecharacteristic amount.

A method of obtaining a candidate point EP in the fourth perspectiveimage (side image) C4 at the time of the therapy, which corresponds tothe specified point P of the second perspective image (side image) C2 atthe time of the therapeutic planning is also the same as the descriptiongiven above. The specified point P of the second perspective image (sideimage) C2 to be used at this time is the candidate point EPcorresponding to the specified point P of the first perspective image(front image) C1. The specified point may be specified either on C3 orC4.

The generating unit 13 enlarges the square image of a predetermined sizehaving a center at the candidate point EP in the second perspectiveimage C2 and generates enlarged images E21, E22, and E23, and alsoenlarges square images of predetermined sizes having centersrespectively at a plurality of candidate points EP within the fourthperspective image C4 and generates enlarged images E41, E42, and E43 asillustrated in FIG. 5, thereby displaying the respective candidatepoints EP in a superimposed manner in the display 14. The display doesnot display the epipolar line L on the second perspective image C2.

The generating unit 13 generates an enlarged image E11, which is anenlarged square image of a predetermined size having a center at thespecified point P within the first perspective image C1 and an enlargedimage E31, which is an enlarged square image of a predetermined sizehaving a center at the candidate point EP within the third perspectiveimage C3, and displays the specified point P and the candidate point EPin the display 14 in a superimposed manner.

The generating unit 13 resizes the square images of predetermined sizesin the first perspective image C1 to the fourth perspective image C4 andgenerates enlarged images. The resizing is performed by general imageresizing methods such as a nearest neighbor method, a bilinear method,and a cubic convolution method.

The process of the medical image processing apparatus 1 will bedescribed on the basis of a flowchart illustrated in FIG. 7.

In Step S1, the first acquiring unit 11 acquires a specified point Pinput into the first perspective image C1.

In Step S2, the second acquiring unit 12 acquires the second perspectiveimage C2 and the first perspective image C1 on which the specified pointP is specified. Subsequently, the second acquiring unit 12 obtains acandidate point EP corresponding to the specified point P on the secondperspective image C2. The second acquiring unit 12 acquires the thirdperspective image C3 and the first perspective image C1 on which thespecified point P is specified. Subsequently, the second acquiring unit12 obtains a candidate point EP corresponding to the specified point Pon the third perspective image C3.

In Step S3, the generating unit 13 generates enlarged images E21, E22,and E23, which are enlarged images of the images in the vicinity of thecandidate point EP within the second perspective image C2. Subsequently,the display 14 displays the enlarged images E21, E22, and E23 togetherwith the candidate point EP. The generating unit 13 generates anenlarged image E31, which is an enlarged image of a portion in thevicinity of the candidate point EP within the third perspective imageC3. Subsequently, the display 14 displays the enlarged image E31together with the candidate point EP. The generating unit 13 generatesenlarged images E41, E42, and E43 in the same manner, and displays thesame together with the candidate point EP in the display 14. The display14 does not display the epipolar line L on the second perspective imageC2.

According to the first embodiment, the candidate points EP on the secondperspective image C2 and the third perspective image C3 corresponding tothe specified point P in the first perspective image C1 are obtainedrespectively, and images of peripheries of the candidate points EP areenlarged to generate the enlarged images E21, E22, E23, and E31 topresent the same to the user, so that convenience at the time of inputof the corresponding point corresponding to the specified point P isimproved. In the fourth perspective image as well, the correspondingpoints can be input easily based on the candidate points EP of theenlarged images E41, E42, and E43.

A first modification of the first embodiment will be described. Theplurality of the perspective images may be images reconstructed from avoxel data (hereinafter, referred to as DRR: Digitally ReconstructedRadiograph), which is data obtained by digitizing the inside of theobject under test 2 obtained at the time of CT photographing from voxelto voxel instead of the X-ray perspective images. Since the voxel datais used, the DRR can be generated by setting view points at givenpositions (the first radiation source S1 and the second radiation sourceS2 illustrated in FIG. 8).

Alternatively, the DRR may be generated from the three-dimensional voxeldata obtained from a PET (position emission tomography) apparatus, or aSPECT (single photon emission computed tomography) apparatus. In thecase where the plurality of perspective images are reversed from eachother between positive and negative, modification is to be performed asneeded.

The plurality of perspective images may be a combination of theperspective images reconstructed from the voxel data obtained by X-rayphotographing, CT-photographing, and the like.

A second modification of the first embodiment will be described. In thesecond acquiring unit 12, the first perspective image C1 and the thirdperspective image C3 are used. However, there is a case where theresolutions of the photographed images are different. In the case wherethe vertical and lateral lengths of one pixel in the three-dimensionalspace of the perspective image are known, the images are resized so thatthe vertical and lateral lengths of the pixel of the two perspectiveimages become the same, and then the candidate points EP are obtained.The resizing of the image is performed by general image resizing methodssuch as a nearest neighbor method, a bilinear method, and a cubicconvolution method as described above.

A third modification of the first embodiment will be described. Thegenerating unit 13 may expand contrast of each enlarged image. Pixeldata of each pixel which constitutes the perspective image has pixeldata (luminance) in a dynamic range wider than the display dynamic range(0 to 255) of the display, which corresponds to the display 14.Therefore, the generating unit 13 expands the contrast so that 256gradations of the display dynamic range are linearly allocated to arange from the minimum value to the maximum value of the pixel values inthe enlarged image. Accordingly, the user can easily find thecorresponding point from the enlarged image. The generating unit 13 maycreate histogram of the pixel data in the enlarged image, and performshistogram equalization to allocate the non-linear gradient.

A fourth modification of the first embodiment will be described. Thegenerating unit 13 may display an enlarged differential image of theenlarged image E11 of the first perspective image C1 and the enlargedimage E21 of the second perspective image C2 when displaying theenlarged image E21. The enlarged differential image may be generatedfrom the differential of luminance values of the pixels each in theenlarged image E11 and the differential of luminance values of therespective pixels in the enlarged image E21. Accordingly, convenience atthe time when the user input the point corresponding to the specifiedpoint P is improved. In the case where the dynamic ranges of the pixeldata of the enlarged image E11 and the enlarged image E22 are different,both of the images are normalized (the range of the normalized data is 0to 1) to obtain the differential (the normalized differential data is −1to 1), and 256 gradations of the display dynamic range are allocated.

A fifth modification of the first embodiment will be described. The usermay directly point out the candidate point EP on the second perspectiveimage C2 by a mouse or the like, and store the pointed positions of thesecond perspective image C2 and the candidate point EP by the secondacquiring unit 12.

A sixth modification of the first embodiment will be described. In thecase where the read perspective image already has a point which has beenspecified at the previous therapy, the point may be stored and reused asthe specified point.

A modification 7 of the first embodiment will be described. In the abovedescription, when the generating unit 13 generates the enlarged image,the enlarged image is centered at the candidate point or the specifiedpoint, however, the invention is not limited thereto, if the candidatepoint or the specified point is included in the enlarged image, thegenerating unit 13 may generate an enlarged image with these pointslocated at positions other than the center.

Second Embodiment

Referring now to FIG. 15 to FIG. 23, the medical image processingapparatus 1 according to a second embodiment will be described.

The medical image processing apparatus 1 according to the secondembodiment will be described on the basis of the block diagramillustrated in FIG. 15. The medical image processing apparatus 1includes the first acquiring unit 11, the second acquiring unit 12, acorrecting unit 21, the generating unit 22, and the display 14. Thesecond embodiment is different from the first embodiment in that thecorrecting unit 21 is added and in the process of the generating unit22. Therefore, only the different configuration and the operation aredescribed, and description of the configuration and the operation whichare the same as those in the first embodiment will be omitted.

The display 14 displays the first perspective image C1, the secondperspective image C2, the third perspective image C3, the fourthperspective image C4, the enlarged image E11, the enlarged image E21,the enlarged image E31, the enlarged image E41, the specified point P,and the candidate point EP as illustrated in FIG. 17 in the same manneras the display 14 of the first embodiment. The display 14 does notdisplay anything if the enlarged images E11, E21, E31, and E41 are notgenerated.

In the following description, the specified point P and the candidatepoint EP are not discriminated, and are expressed simply as “point”.

The generating unit 22 acquires the candidate points EP from the firstacquiring unit 11 and the second acquiring unit 12, generates anenlarged image, and displays the same in the display 14 in the samemanner as the generating unit 13 of the first embodiment. The generatingunit 22 acquires coordinate of the specified point P and the candidatepoint EP corrected by the correcting unit 21, which will be describedlater, and generates the enlarged image and displays the same in thedisplay 14.

The user corrects the positions of the specified point P and thecandidate point EP displayed in the display 14 by using an interfacesuch as a mouse, and the correcting unit 21 acquires the coordinates ofthe corrected points and outputs the coordinates of the corrected pointsto the generating unit 22. In the case where the mouse pointer is on theenlarged images E11, E21, E31, and E41 displayed on the display 14, andthe correcting unit 21 does not select a point, an enlargement factor ischanged by moving the mouse wheel.

The correction of the point by the correcting unit 21 is achieved by theuser, by clicking at a position near the point to select the point witha mouse, and dragging the point to move the position. For example, whenthe user selects a point P11 on the first perspective image C1, thecorrecting unit 21 changes successively the positional coordinate of thepoint P11 on the first perspective image C1, and outputs the coordinatesto the generating unit 22 while drawing the points. The generating unit22 changes the enlarged image E11 into an enlarged image E11 having acenter at the point P11 or its coordinates, at that time point. At thistime, since the drawing position of the point EP11 is located at thecenter of the enlarged image E11, there is no more change in the drawingposition. Also, the correcting unit 21 does not change the magnitude ofthe display range of the enlarged image E11.

When the user selects and corrects the point EP11 on the enlarged imageE11, the correcting unit 21 does not output the coordinates of the pointEP11, which is changed successively, to the generating unit 22, andchanges only the drawing positions of the point EP11 on the enlargedimage E11 and the point P11 on the first perspective image C1successively.

The correcting unit 21 changes the enlarged image in the same manneralso in the case where the point EP21, the point EP31, and the pointEP41 are selected.

The process of the medical image processing apparatus 1 will bedescribed on the basis of the flowchart illustrated in FIG. 16. Theflowchart is started from the time point when the enlarged image isgenerated in the first embodiment.

In Step S11, the generating unit 22 displays an enlarged image and acandidate point EP in the display 14 as illustrated in FIG. 17.

In Step S12, the correcting unit 21 corrects the coordinate of thespecified point P or the candidate point EP selected and corrected bythe user.

In Step S13, the generating unit 22 generates an enlarged imageaccording to the coordinate of the corrected point, and displays againin the display 14.

According to the second embodiment, since the user can easily correctthe points on the image, input of the corresponding point can beperformed further easily.

A first modification of the second embodiment will be described. Asillustrated in FIG. 18, the horizontal length and vertical length in thethree-dimensional space of the display range of the enlarged image maybe displayed respectively on the lower side and on the right side ofeach of the enlarged images E11, E21, E31, and E41, in accordance withthe magnification factors of the enlarged images E11, E21, E31, and E41.

A second modification of the second embodiment will be described. Asillustrated in FIG. 19, a part or a range of the first perspective imageC1, which is displayed in the enlarged image E11 may be indicated by arectangular frame B11 on the first perspective image C1. The sameapplies to the second perspective image C2, the third perspective imageC3, and the fourth perspective image C4.

A third modification of the second embodiment will be described. In thismodification, manner of displaying on the display 14 will be describedin the case where there are a plurality of the candidate points EP. Asillustrated in FIG. 20, the display 14 shows a plurality of candidatepoints EP31 and EP32 on the third perspective image C3 in relation to aspecified point P11 of the first perspective image C1. At this time, theuser clicks one of the enlarged images E31 and E32, and selects one ofthe plurality of candidate points EP. When the selection is completed,the display goes to a screen illustrated to FIG. 17. In FIG. 20, since acolor coding display is not possible in the drawings, differentiatedhatching lines are employed.

The scales as illustrated in FIG. 18 may be displayed on the lower sideand the right side of the enlarged images E11, E21, and E13,respectively (not illustrated).

If there is a plurality of candidate points EP, the respective pointsP31 and P32 may be color coded and displayed as indicated in FIG. 21. Itis also possible to provide the frames in the same colors as those usedfor indicating the points, on fringes of the enlarged images E31 and E32corresponding to the respective candidate points EP. In FIG. 21, since acolor coding display is not possible in the drawings, differentiatedhatching lines are employed.

A fourth modification of the second embodiment will be described. In theabove-described embodiments, the enlarged images E11, E21, E31, and E41are arranged on the right side of the window W1. However, these may bearranged on the left side of the window W1 (not illustrated).Alternatively, as illustrated in FIG. 22 and FIG. 23, the enlargedimages may be displayed with floating windows W2 and W02.

A fifth modification of the second embodiment will be described. Toolsused for selection and transfer of points by the correcting unit 21 arenot limited to the mouse and a keyboard and a touch pen may be employedand, if the display has a touch panel, a flick input by the user is alsoapplicable.

In the respective embodiments described above, description has beengiven with the perspective images which are obtained by photographingthe first perspective image C1 and the second perspective image C2 atthe time of the therapeutic planning and photographing the thirdperspective image C3 and the fourth perspective image C4 at the time ofthe therapy. However, the invention is not limited thereto, and theperspective images may be obtained by photographing the firstperspective image C1 and the second perspective image C2 at the time ofthe first therapy, and photographing the third perspective image C3 andthe fourth perspective image C4 at the time of the second therapy. Inother words, what is essential is that the first perspective image C1and the second perspective image C2 are photographed at thesubstantially same time and the third perspective image C3 and thefourth perspective image C4 are photographed at the substantially sametime, and the first perspective image C1 and the second perspectiveimage C2 are photographed at different time or at the substantially sametime as the third perspective image C3 and the fourth perspective imageC4.

In the above-described embodiment, the front image and the side image ofthe object under test 2 are used as the perspective images photographedfrom the two different directions. However, this is an example only, andthe other perspective images of the object under test 2 from the twodifferent directions are also applicable.

In the above-described embodiment, the head region of the patient hasbeen described as the object under test 2. However, the invention is notlimited thereto, other portions such as a breast region or an abdominalregion are also applicable.

The medical image processing apparatus 1 may be implemented by using,for example, a general purpose computer apparatus as a basic hardware.In other words, the first acquiring unit 11, the second acquiring unit12, the generating unit 13, the correcting unit 21, and the generatingunit 22 are implemented by causing a processor mounted on theabove-described computer apparatus to execute a program. At this time,the medical image processing apparatus 1 may be implemented byinstalling the above-described problem in the computer apparatus inadvance, or may be implemented by storing the same in a storage mediumsuch as a CD-ROM or an USB memory, distributing the above-describedprogram via network, and installing the distributed program in thecomputer apparatus as needed. The storage device for the perspectiveimages or the like in the medical image processing apparatus 1 may beimplemented by using integrated or external memory, or a hard disk, orstorage medium such as a CD-R, a CD-RW, a DVD-RAM, a DVD-R, as needed.

FIG. 24 is a block diagram showing an example of hardware constructionof the medical image processing apparatus. As shown herein, the medicalimage processing apparatus 1 comprises: a CPU (central processing unit)101; an ROM (Read Only Memory) 102 storing the program for measurement,and the like; RAM (Random Access Memory) 103; HDD (hard disk drive) 104;I/F 105, which is an interface for the HDD 104; a photographingapparatus 106 having radiation sources S1 and S2 such as X-rayphotographing apparatus, a CT scanner and an MRI (Magnetic ResonanceImaging) apparatus; I/F 107, which is an interface for the photographingapparatus; an input device 108 such as a mouse or a keyboard; I/F 109,which is an interface for the input device; a display 14 such as avisual display unit; I/F 110, which is an interface for the display 14;a computer bus 111. Thus, the medical image processing apparatus 1utilizes a hardware construction of common computers. The CPU 101, theROM 102, the RAM 103, the I/Fs 105, 107, 109 and 110 are connected witheach other, through the computer bus 111.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

What is claimed is:
 1. An apparatus for medical image-based radiotherapyplanning, comprising: a processor; and a memory configured to storeprocessor-executable instructions which, when executed by the processor,cause the processor to at least: receive a first perspective image and asecond perspective image, obtained by photographing an object fromdifferent directions; acquire positional information of a specifiedpoint on the first perspective image; search, in a searching rangeincluding an epipolar line on the second perspective image, for acandidate point based on a Haar-like characteristic amount, thecandidate point corresponding to the specified point, the epipolar linecorresponding to the specified point; calculate the Haar-likecharacteristic amount of a given point in the searching range byrotating a Haar-like characteristic calculation window having a centerat a given position in accordance with an inclination of the epipolarline; and generate a first enlarged image by enlarging a part of thefirst perspective image neighboring the specified point, a secondenlarged image by enlarging a part of the second perspective imageneighboring the candidate point, and a differential image indicatingdifferences between luminance values of pixels in the first enlargedimage and luminance values of pixels in the second enlarged image. 2.The apparatus according to claim 1, wherein the instructions cause theprocessor to generate the second enlarged image when the candidate pointexists on the second perspective image.
 3. The apparatus according toclaim 1, wherein the instructions cause the processor to determine apoint having a highest Haar-like characteristic amount as the candidatepoint.
 4. The apparatus according to claim 1, wherein the instructionscause the processor to determine points from a point having a highestHaar-like characteristic amount to a point having an Nth highestHaar-like characteristic amount as candidate points, wherein N>1.
 5. Theapparatus according to claim 1, wherein the instructions cause theprocessor to divide a searching range into divided areas, and search forcandidate points in each divided area.
 6. The apparatus according toclaim 1, wherein the instructions cause the processor to generate thefirst and second enlarged images superimposed on the candidate point. 7.The apparatus according to claim 6, wherein the instructions cause theprocessor to process a received instruction to move the candidate pointon the second enlarged image.
 8. The apparatus according to claim 7,wherein the instructions cause the processor to not change a size of adisplay range of the first and second enlarged images when the candidatepoint is moved.
 9. The apparatus according to claim 6, wherein theinstructions cause the processor to move the candidate point on thesecond enlarged image so that a corrected candidate point moves onto theepipolar line.
 10. The apparatus according to claim 1, wherein theinstructions cause the processor to determine a point at which theHaar-like characteristic amount is a maximum as a position of thecandidate point.
 11. The apparatus according to claim 1, wherein theinstructions cause the processor to resize the second perspective imageand generate the second enlarged image.
 12. The apparatus according toclaim 1, further comprising a display, wherein the display displays thefirst perspective image, the second perspective image, and first andsecond enlarged images on a screen.
 13. A method for medical image-basedradiotherapy planning, comprising: receiving a first perspective imageand a second perspective image, obtained by photographing an object fromdifferent directions; acquiring positional information of a specifiedpoint on the first perspective image; searching, in a searching rangeincluding an epipolar line on the second perspective image, for acandidate point based on a Haar-like characteristic amount, thecandidate point corresponding to the specified point, the epipolar linecorresponding to the specified point; calculating the Haar-likecharacteristic amount of a given point in the searching range byrotating a Haar-like characteristic calculation window having a centerat a given position in accordance with an inclination of the epipolarline; and generating a first enlarged image by enlarging a part of thefirst perspective image neighboring the specified point, a secondenlarged image by enlarging a part of the second perspective imageneighboring the candidate point, and a differential image indicatingdifferences between luminance values of pixels in the first enlargedimage and luminance values of pixels in the second enlarged image.
 14. Asystem for medical image-based radiotherapy planning, comprising: amemory device for storing computer readable program code; and aprocessor in communication with the memory device, the processor beingoperative with the computer readable program code to at least: receive afirst perspective image and a second perspective image, obtained byphotographing an object from different directions; acquire positionalinformation of a specified point on the first perspective image; search,in a searching range including an epipolar line on the secondperspective image, for a candidate point based on a Haar-likecharacteristic amount, the candidate point corresponding to thespecified point, the epipolar line corresponding to the specified point;calculate the Haar-like characteristic amount of a given point in thesearching range by rotating a Haar-like characteristic calculationwindow having a center at a given position in accordance with aninclination of the epipolar line; and generate a first enlarged image byenlarging a part of the first perspective image neighboring thespecified point, a second enlarged image by enlarging a part of thesecond perspective image neighboring the candidate point, and adifferential image indicating differences between luminance values ofpixels in the first enlarged image and luminance values of pixels in thesecond enlarged image.
 15. The system according to claim 14, furthercomprising a radiation source which emits radiation directed at a targetin the object.